Patient satisfaction surveys fail to track how well hospitals treat people of color

Each day, thousands of patients get a call or letter after being discharged from U.S. hospitals. How did their stay go? How clean and quiet was the room? How often did nurses and doctors treat them with courtesy and respect?

The questions focus on what might be termed the standard customer satisfaction aspects of a medical stay, as hospitals increasingly view patients as consumers who can take their business elsewhere.

But other crucial questions are absent from these ubiquitous surveys, whose results influence how much hospitals get paid by insurers: They do not poll patients on whether they've experienced discrimination during their treatment, a common complaint of diverse patient populations.

Likewise, they fail to ask diverse groups of patients whether they've received culturally competent care.

And some researchers say that's a major oversight.

Kevin Nguyen, a health services researcher at Brown University School of Public Health, who parsed data collected from the government-mandated national surveys in new ways, found that — underneath the surface — they spoke to racial and ethnic inequities in care.

Digging deep, Nguyen studied whether patients in one Medicaid managed-care plan from ethnic minority groups received the same care as their white peers. He examined four areas: access to needed care, access to a personal doctor, timely access to a checkup or routine care, and timely access to specialty care.

"This was pretty universal across races. So Black beneficiaries; Asian American, Native Hawaiian, and Pacific Islander beneficiaries; and Hispanic or Latino or Latinx/Latine beneficiaries reported worse experiences across the four measures," he said.

Nguyen said that the surveys commonly used by hospitals (called Consumer Assessment of Healthcare Providers and Systems, or CAHPS) could be far more useful if they were able to go one layer deeper — for example, asking why it was more difficult to get timely care, or why they don't have a personal doctor.

It would also be more helpful if CMS publicly posted not just the aggregate patient experience scores, but also showed how those scores varied by respondents' race, ethnicity, and preferred language.

Such data can help discover whether a hospital or health insurance plan is meeting the needs of all versus only some patients. Nguyen did not study responses of LGBTQ+ individuals or, for example, whether people received worse care because they were obese.

Hospital surveys — and how to game them — has become big business

The health care provider surveys are required by the federal government for many health care facilities, and the hospital version of it is required for most acute care hospitals. Low scores can induce financial penalties, and hospitals reap financial rewards for improving scores or exceeding those of their peers.

The CAHPS Hospital Survey, known as HCAHPS, has been around for more than 15 years. The results are publicly reported by the Centers for Medicare & Medicaid Services to give patients a way to compare hospitals, and to give hospitals incentive to improve care and services. Patient experience is just one thing the federal government publicly measures; readmissions and deaths from conditions including heart attacks and treatable surgery complications are among the others.

Dr. Meena Seshamani, director of the Center for Medicare, said that patients in the U.S. seem to be growing more satisfied with their care:

"We have seen significant improvements in the HCAHPS scores over time," she said in a written statement, noting, for example, that the percentage of patients nationally who said their nurses "always" communicated well rose from 74% in 2009 to 81% in 2020.

But for as long as these surveys have been around, doubts about what they really capture have persisted. Patient experience surveys have become big business, with companies marketing methods to boost scores. Researchers have questioned whether the emphasis on patient satisfaction — and the financial carrots and sticks tied to them — have led to better care. And they have long suspected institutions can "teach to the test" by training staff to cue patients to respond in a certain way.

National studies have found the link between patient satisfaction and health outcomes is tenuous at best. Some of the more critical research has concluded that "good ratings depend more on manipulable patient perceptions than on good medicine," citing evidence that health professionals were motivated to respond to patients' requests rather than prioritize what was best from a care standpoint, when they were in conflict.

Hospitals have also scripted how nurses should speak to patients to boost their satisfaction scores. For example, some were instructed to cue patients to say their room was quiet by making sure to say out loud, "I am closing the door and turning out the lights to keep the hospital quiet at night."

A new push to survey hospitals about discrimination

About a decade ago, Robert Weech-Maldonado, a health services researcher at the University of Alabama-Birmingham, helped develop a new module to add to the HCAHPS survey "dealing with things like experiences with discrimination, issues of trust." Specifically, it asked patients how often they'd been treated unfairly due to characteristics like race or ethnicity, the type of health plan they had (or if they lacked insurance), or how well they spoke English.

It also asked patients if they felt they could trust the provider with their medical care. The goal, he said, was for that data to be publicly reported, so patients could use it.

Some of the questions made it into an optional bit of the HCAHPS survey — including questions on how often staffers were condescending or rude, and how often patients felt the staff cared about them as a person — but CMS doesn't track how many hospitals use them, or how they use the results. And though HCAHPS asks respondents about their race, ethnicity and language spoken at home, CMS does not post that data on its public patient website, nor does it show how patients of various identities responded compared to others.

Without wider use of explicit questions about discrimination, Dr. Jose Figueroa, an assistant professor of health policy and management at the Harvard School of Public Health, doubts HCAHPS data alone would "tell you whether or not you have a racist system" — especially given the surveys' slumping response rates.

One exciting development, he said, lies with the emerging ability to analyze open-ended (rather than multiple-choice) responses through what's called natural language processing, which uses artificial intelligence to analyze the sentiments people express in written or spoken statements as an addendum to the multiple-choice surveys.

One study analyzing hospital reviews on Yelp identified characteristics patients think are important but aren't captured by HCAHPS questions — like how caring and comforting staff members were, and the billing experience. And a study out this year in the journal Health Affairs used the method to discover that providers at one medical center were much more likely to use negative words when describing Black patients compared with their white counterparts.

"It's simple, but if used in the right way can really help health systems and hospitals figure out whether they need to work on issues of racism within them," said Figueroa.

Press Ganey Associates, a company that a large number of U.S. hospitals pay to administer these surveys, is also exploring this idea. Dr. Tejal Gandhi leads a project there that, among other things, aims to use artificial intelligence to probe patients' comments for signs of inequities.

"It's still pretty early days," Gandhi said, adding, "With what's gone on with the pandemic, and with social justice issues, and all those things over the last couple of years, there's just been a much greater interest in this topic area."

Direct outreach to improve cultural competence

Some hospitals, though, have taken the tried-and-true route to understanding how to better meet patients' needs: talking to them.

Dr. Monica Federico, a pediatric pulmonologist at the University of Colorado School of Medicine and Children's Hospital Colorado in Denver, started an asthma program at the hospital several years ago. About a fifth of its appointments proved no-shows. The team needed something more granular than patient satisfaction data to understand why.

"We identified patients who had been in the hospital for asthma, and we called them, and we asked them, you know, 'Hey, you have an appointment in the asthma clinic coming up. Are there any barriers to you being able to come?' And we tried to understand what those were," said Federico.

At the time, she was one of the only Spanish-speaking providers in an area where pediatric asthma disproportionately affects Latino residents. (Patients also cited problems with transportation and inconvenient clinic hours.)

After making several changes, including extending the clinic's hours into the evening, the no-show appointment rate nearly halved.

Patient satisfaction surveys are embedded in American health care culture and are likely here to stay. But CMS is now making tentative efforts in surveys to address the issues that were previously overlooked: As of this summer, it is testing a question for a subset of patients 65 and older that would explicitly ask if anyone from a clinic, emergency room, or doctor's office treated them "in an unfair or insensitive way" because of characteristics including race, ethnicity, culture, or sexual orientation.

KHN (Kaiser Health News) is a national newsroom that produces in-depth journalism about health issues. It is an editorially independent operating program of KFF (Kaiser Family Foundation). KFF is an endowed nonprofit organization providing information on health issues to the nation.